Causal Models: the Meaningful Information of Probability Distributions

This paper claims that causal model theory describes the meaningful information of probability distributions after a factorization. If the minimal factorization of a distribution is incompressible, its Kolmogorov minimal sufficient statistics, the parents lists, can be represented by a directed acyclic graph (DAG). We showed that a faithful Bayesian network is a minimal factorization and that a Bayesian network with random and unrelated conditional probability distributions (CPDs) is faithful and thus a minimal factorization. The validity of faithfulness depends on the presence of other regularities. The Bayesian network is a canonical representation, it uniquely decomposes the distribution into independent submodels, the CPDs. In absence of further information, we may assume modularity and that the model offers a good hypothesis about the underlying mechanisms of the system.